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Management  Uittreksel 

De  doelstelling  van  deze  studie  was  de  overeenkomsten  en  de  verschillen  die  er 
tussen  de  verschillende  typen  cognitieve  taakanalyses  bestaan  na  te  gaan  en,  indien 
mogelijk,  een  theoretisch  raamwerk  af  te  leiden,  waarbinnen  de  leemten  op 
onderzoeksgebied  zichtbaar  worden.  Het  resultant  van  deze  studie  is  een  theoretisch 
raamwerk  dat  drie  perspectieven  op  cognitieve  taakanalyse  integreert:  taak- 
modellering,  kennismodellering  en  cognitieve  modellering.  Elk  der  drie  perspectieven 
wordt  respectievelijk  uitgelegd  aan  de  hand  van  drie  prototypische  taakanalyse 
technieken:  Hierarchical  Task  Analysis  (HTA),  Knowledge  Analysis  and  Documen¬ 
tation  System  (KADS)  and  Goals  Operators  Methods  and  Selection  rules  (GOMS). 

De  perspectieven  op  taakmodellering  en  cognitieve  modeliering  zijn  reeds  bekend  in 
onderzoekskringen  van  cognitieve  taakanalyse.  De  taakmodellering  is  gericht  op  de 
decompositie  van  taken  in  doelen  en  subdoelen.  De  cognitieve  modellering  con- 
centreert  zich  op  de  cognitieve  performance  van  de  taak.  Het  perspectief  op 
kennismodellering  daarentegen  is  voor  het  onderzoeksgebied  van  de  cognitieve 
taakanalyse  een  relatief  nieuw  element. 

Binnen  het  perspectief  van  de  kennismodellering  wordt  informatie  verzameld  over  de 
vereisten  van  de  taak;  met  name  over  de  kennis  en  de  strategieen  die  nodig  zijn  om 
de  taakdoelen  te  bereiken.  Deze  taakvereisten  kunnen  een  sterke  invloed  uitoefenen 
op  de  cognitieve  performance  van  de  taak.  Vandaar  dat  birmen  het  raamwerk  voor 
cognitieve  taakanalyse  wordt  aanbevolen  de  taakvereisten  te  onderzoeken  alvorens  te 
beginnen  met  het  analyseren  van  de  cognitieve  performance  van  de  taak.  Vele  typen 
cognitieve  taakanalyses  richten  zich  te  sterk  op  de  cognitieve  performance  van  de 
taak  zonder  veel  aandacht  te  besteden  aan  de  vereisten  van  de  taak. 

Verder  onderzoek  op  dit  gebied  is  dan  ook  nudig  om  een  brug  te  slaan  tussen  de 
twee  kampen  van  taakmodellering  enerzijds  en  cognitieve  modellering  anderzijds.  Een 
interessant  object  voor  onderzoek  zou  kunnen  bestaan  uit  het  ontwikkelen  van  een 
wijze  om  de  vereisten  van  een  taak  op  zodanige  wijze  te  verkrijgen  dat  op  grond  van 
deze  taakvereisten,  samen  met  de  verkregen  taakdoelen,  voorspellingen  kunnen 
worden  gedaan  over  de  cognitieve  performance. 


*Per  1  februari  1994  i$  de 
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SUMMARY 

The  goal  of  this  study  was  to  examine  similarities  and  differences  between 
various  types  of  cognitive  task  analysis  and,  if  possible,  to  infer  some  framework 
in  which  the  areas  for  further  research  should  become  clear.  The  result  of  this 
study  is  a  framework  which  integrates  three  views  on  cognitive  task  analysis; 
task,  knowledge  and  cognitive  modelling.  For  each  view,  prototypical  task 
analysis  techniques  are  presented  respectively:  Hierarchical  Task  Analysis 
(HTA),  Knowledge  Analysis  and  Documentation  System  (KADS)  and  Goals 
Operators  Methods  and  Selection  rules  (GOMS). 

The  task  modelling  view  and  the  cognitive  modelling  view  are  already  known  in 
the  cognitive  task  analysis  domain.  The  task  modelling  view  concentrates  on  the 
decomposition  of  tasks  into  goals  and  subgoals.  The  cognitive  modelling  view 
concentrates  on  the  cognitive  performance  of  the  task.  The  knowledge  modelling 
view,  however,  is  a  relatively  new  element  in  the  research  area  of  cognitive  task 
analysis. 

Within  the  knowledge  modelling  view,  it  is  recommended  by  the  framework  to 
gain  information  about  the  requirements  of  the  task.  Those  requirements  can 
have  a  strong  influence  on  the  cognitive  performance  of  the  task  and,  therefore, 
within  the  framework  of  cognitive  task  analysis,  it  is  advised  to  examine  task 
requirements  before  starting  to  analyze  the  cognitive  performance  of  the  task. 
Many  types  of  cognitive  task  analysis  focus  too  much  on  the  cognitive  perform¬ 
ance  of  a  task,  without  paying  much  attention  to  the  requirements  of  the  task. 
Further  research  in  this  area  is  therefore  necessary  to  bridge  the  gap  between 
one  sight  of  the  camp,  the  task  modelling  view,  and  the  other  sight  of  the  camp, 
the  cognitive  modelling  view.  An  interesting  work  of  research  could  be  to  invent 
a  way  in  which  task  requirements  are  acquired  in  such  a  way  that  they,  together 
with  the  task^  goals,  can  easily  be  used  to  make  predictions  about  cognitive 
performance. 
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SAMENVATTING 

De  doelstelling  van  deze  studie  was  de  overeenkomsten  en  de  verschillen  die  er 
tussen  de  verschillende  typen  cognitieve  laakanalyses  bestaan  na  te  gaan  en, 
indien  mogelijk,  een  theoretisch  raamwerk  af  te  leiden,  waarbinnen  de  leemten 
op  onderzoeksgebied  zichtbaar  worden.  Het  resuUaat  van  deze  studie  is  een 
theoretisch  raamwerk  dat  drie  perspectieven  op  cognitieve  taakanalyse  inte- 
greert:  taakmodellering,  kennismodeilering  en  cognitieve  modeliering.  Elk  der 
drie  perspectieven  wordt  respectievelijk  uitgelegd  aan  de  hand  van  drie  proto- 
typische  taakanalyse  technieken:  Hierarchical  Task  Analysis  (HTA),  Knowledge 
Analysis  and  Documentation  System  (KADS)  and  Goals  Operators  Methods  and 
Selection  rules  (GOMS). 

De  perspectieven  op  taakmodellering  en  cognitieve  modeliering  zijn  reeds 
bekend  in  onderzoekskringen  van  cognitieve  taakanalyse.  De  taakmodellering  is 
gericht  op  de  decompositie  van  taken  in  doelen  en  subdoelen.  De  cognitieve 
modellering  concentreert  zich  op  de  cognitieve  performance  van  de  taak.  Het 
perspectief  op  kennismodeilering  daarentegen  is  voor  het  onderzoeksgebied  van 
de  cognitieve  taakanalyse  een  reSatief  meuw  element. 

Biimen  het  perspectief  van  de  kennismodeilering  wordt  informatie  verzameld 
over  de  vereisten  van  de  taak;  met  name  over  de  kennis  en  de  strategieen  die 
nodig  zijn  om  de  taakdoelen  te  bereiken.  Deze  taakvereisten  kunnen  een  sterke 
invloed  uitoefenen  op  de  cognitieve  performance  van  de  taak.  Vandaar  dat 
binnen  het  raamwerk  voor  cognitieve  taakanalyse  wordt  aanbevolen  de  taak¬ 
vereisten  te  onderzoeken  alvorens  te  beginnen  met  het  analyseren  van  de 
cognitieve  performance  van  de  taak.  Vele  typen  cognitieve  taakanalyses  richten 
zich  te  sterk  op  de  cognitieve  performance  van  de  taak  zonder  veel  aandacht  te 
besteden  aan  de  vereisten  van  de  taak. 

Verder  onderzoek  op  dit  gebied  is  dan  ook  nodig  om  een  brug  te  slaan  tussen  de 
twee  kampen  van  taakmodellering  enerzijds  en  cognitieve  modellering  ander- 
zijds.  Een  interessant  object  voor  onderzoek  zou  kunnen  bestaan  uit  het  ontwik- 
kelen  van  een  wijze  om  de  vereisten  van  een  taak  op  zodanige  wijze  te  ver- 
krijgen  dat  op  grond  van  deze  taakvereisten,  samen  met  de  verkregen  taak- 
doeien,  voorspellingen  kunnen  worden  gedaan  over  de  cognitieve  performance. 


*Per  1  februari  1994  is  ile  aaam  Instituut  voor  Zincuigfysiologie  TNO  gewijzigd  m  TNO  Technische 
Menskunde. 
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1  INTRODUCTION 

The  goal  of  the  present  report  is  threefold;  firstly,  to  present  a  framework  of 
cognitive  task  analysis  unifying  the  often  isolated  techniques  found  in  the 
literature;  secondly,  to  assign  different  techniques  for  cognitive  task  analysis  to  a 
limited  number  of  categories;  thirdly,  to  illustrate  the  use  of  cognitive  task 
analysis  in  practice.  Until  now,  cognitive  task  analysis  techniques  have  been 
developed  ad*hoc,  without  a  framework.  This  has  caused  so  much  chaos,  that  a 
clear  state-of-the-art  review  of  cognitive  task  analysis  cannot  be  given.  Therefore, 
first  a  unifying  framework  of  cognitive  task  analysis  had  to  be  developed.  The 
description  of  cognitive  task  analysis  in  this  report  is  oriented  more  towards 
applications  than  towards  fundamental  research.  This  means  that  methods  of 
cognitive  task  analysis  are  only  described  in  relation  to  theories  of  cognitive 
psychology,  if  results  of  cognitive  psychological  research  pl^  an  important  role 
in  those  methods.  The  analysis  techniques  which  are  described  focus  on  complex, 
cognitive  tasks  which  require  a  higher  level  of  information  processing  from  the 
task  performer  than  relatively  simple,  physical  tasks. 

The  global  structure  of  the  report  can  be  viewed  as  follows.  Chapter  two 
describes  the  genesis  of  cognitive  task  analysis  out  of  traditional  task  analysis. 
Chapter  three  describes  the  framework  which  we  have  developed  to  elucidate 
the  area  of  cognitive  task  analysis.  This  chapter  is  divided  into  three  paragraphs 
each  describing  the  three  views  on  cognitive  task  analysis:  task  modelling, 
knowledge  modelling  and  cognitive  modelling.  Chapter  four  describes  three 
applications  corresponding  to  the  three  modelling  views.  These  modelling  views 
are  described  according  to  pratotypical  applications:  the  Hierarchical  Task 
Analysis  as  a  representative  of  task  modelling,  the  KADS  methodology  of 
knowledge  modelling  and  the  GOMS  methodology  of  cognitive  modelling. 
Chapter  five  describes  the  integration  of  the  three  views  into  cognitive  task 
analysis.  Again  this  chapter  is  divided  into  two  paragraphs:  the  first  describes  the 
argumentation  behind  the  integration  of  cognitive  task  analysis;  the  second 
describes  the  practical  consequences  of  using  cognitive  task  analysis.  The  last 
chapter  contains  a  general  discussion  of  the  usability  of  this  cognitive  task 
analysis  approach  and  gives  some  suggestions  for  further  research. 


2  WHY  COGNITIVE  TASK  ANALYSIS? 

During  the  last  three  decades,  task  environments,  task  requirements  and  human 
performance  were  analysed  to  predict  or  detect  problems  in  task  performance 
and  to  render  solutions  to  these  problems.  The  kind  of  analysis  that  is  meant 
here  is  called  task  analysis  (Drury,  Paramore,  Van  Cott,  Grey  &  Corlett,  1987; 
Kirwan  &  Ainsworth,  1992).  Task  analysis  was  used  for  various  purposes:  to 
identify  the  performance  demands  of  tasks,  to  identify  the  skills  and  knowledge 
needed  to  perform  the  tasks,  to  calculate  the  number  of  persons  needed  to  carry 
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out  a  task  or  to  design  the  interface  between  human  performers  and  equipment. 
Good  progress  has  been  made  on  these  topics.  Depending  on  the  desired 
purpose  of  the  task  analysis  a  particular  technique  was  chosen.  Despite  the  great 
variability  among  task  analysis  techniques,  the  techniques  did  have  one  impor¬ 
tant  thing  in  common:  they  all  studied  observable  behaviour  of  human  perform¬ 
ers.  Because  the  overt  behaviour  of  human  performers  often  does  not  include 
signs  of  actual  cognitive  activity  but  rather  mirrors  its  results,  these  techniques 
concentrate  more  on  the  results  of  the  process  than  on  the  nature  of  the  process 
itself.  This  concentration  on  the  observable  behaviour  and  thus  the  results  of  the 
process,  however,  made  task  analysis  techniques  less  usable  since  the  period  in 
which  automation  of  the  workplace  took  place  (Moray,  Sanderson  &  Vicente, 
1992).  This  increasing  introduction  of  automation  into  the  workplace  has 
changed  the  task  environments,  task  requirements  and  human  performance  so 
dramatically  that  the  call  for  a  new  kind  of  task  analysis  became  urgent:  the  so- 
called  cognitive  task  analysis. 

Replacing  human  task  performers  by  automation  at  the  workplace  has  speeded 
up  the  call  for  a  cognitive  task  analysis  for  the  following  reasons. 

Firstly,  routine  skills  were  taken  over  by  machines.  As  a  consequence,  maity 
human  performers  were  no  longer  operating  on  the  task  objects  and  the  task 
process  itself  but  were  more-and-more  supervising  task  execution  by  machines 
(Moray  et  al.,  1992).  This  change  can  be  regarded  as  a  change  from  pl^sical 
manipulation  to  mental  decision  making  in  human  performers.  At  the  lowest 
level  of  automation  of  a  system,  human  performers  directly  control  the  hardware 
and  monitor  its  performance  parameters  and  results  by  means  of  displays  of 
sensor  outputs,  by  direct  perception  or  by  a  combination  of  these  (Drury  et  al., 
1987),  With  higher  levels  of  automation,  human  performance  requirements 
become  increasingly  supervisory  in  nature,  forcing  the  human  performers  to 
continually  supervise  the  processing  of  the  system  until  a  failure  occurs.  This 
failure  must  be  recognized,  interpreted  and  compensated  for  by  the  human 
performer,  often  in  a  relatively  short  time  limit.  In  these  situatioas,  little  empha¬ 
sis  is  placed  upon  the  performer’s  physical  manipulation  activities  as  opposed  to 
the  performer’s  mental  decision  making  activities.  These  mental  decision  making 
activities  are  best  analysed  by  a  task  analysis  which  is  oriented  towards  non¬ 
observable,  cognitive  activities. 

Secondly,  automation  has  increased  and  accelerated  the  stream  of  information  in 
workplaces.  Infijrmation,  which  is  at  the  same  time  more  complex  and  more 
diftiise  (coming  from  very  different  directions),  has  to  be  processed  by  human 
performers  as  quickly  as  before  or  even  more  quickly  (Helander,  1988;  Wickens, 
1992).  In  this  context,  Wickens  describes  the  example  of  the  transition  from 
driving  an  oxcart  vehicie  to  navigating  a  spacecraft.  To  the  oxcart  driver,  a 
fraction  of  a  second  delay  in  responding  to  an  environmental  event  will  be  of 
little  consequence.  But  to  the  navigator  of  a  spacecraft,  a  delay  of  the  same 
absolute  magnitude  may  be  critical  in  causing  a  collision.  Considering  the 
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increase  in  the  maximum  speed  of  both  vehicles  and  the  increasing  chance  of 
errore  in  human  performance,  tl’ese  developments  make  it  very  urgent  to  anatyse 
the  operator’s  cognitive  processes  quite  closely,  such  as  the  operatorls  temporal 
limits  of  processing  information. 

Finally,  not  only  routine  skills  but  complex  problem  solving  as  well,  is  sometimes 
taken  wer  by  machines  (Drury  et  al.,  1987).  Cognitive  tasks  of  humans  have 
been  taken  over  by  knowledge  based  systems,  such  as  expert  systems,  intelligent 
tutoring  systems  and  computer  simulations.  These  systems  were  developed  to 
support  htiman  performance.  In  order  to  develop  those  problem  solving 
machines  very  intensive  studies  of  the  cognitive  tasks  were  required. 

To  summarise  the  aspects  mentioned  above,  conventional  task  analysis  is  limited 
in  preparing  guidelines  for  designing  systems  that  will  help  operators  as  they 
make  judgments  and  decisions.  Automation  of  the  workplace  has  increased  the 
demand  on  cognitive  skills  of  human  performers.  Especially  in  the  domain  of 
system  design,  the  change  in  focus  on  cognitive  tasks  is  accompanied  by  an 
increasing  need  for  techniques  with  which  these  cognitive  tasks  can  be  analysed. 
These  cognitive  task  analyses  should  allow  to  analyse  complex  tasks  in  which 
uncertain  and  dynamic  data  must  be  interpreted  and  assessed.  This  explains  the 
main  reasons  why  a  cognitive  task  analysis,  which  is  concentrated  more  on  the 
nature  of  the  process  than  on  the  results  of  the  process,  was  strongly  asked  for. 


3  FRAMEWORK 

Now  that  the  importance  of  cognitive  task  analysis  for  various  domains  has  been 
clarified,  the  question  arises  what  is  understood  by  cognitive  task  analysis.  A 
final  answer  to  this  question  cannot  easily  be  given,  because  a  unique  description 
of  cognitive  task  analysis  cannot  be  found  in  the  literature.  In  searching  through 
the  literature,  it  became  apparent  that  the  term  "cognitive  task  analysis"  is  used 
in  various  contexts,  from  different  perspectives  and  with  variable  meanings.  The 
literature  search  was  done  on  the  basis  of  the  following  keywords;  cognitive 
processes,  task  analysis,  cognitive  task  analysis,  cognitive  model,  methodology 
and  combinations  of  these  keywords.  The  techniques  retrieved  by  the  literature 
search  are  described  in  the  appendix.  A  framework  that  normally  represents  the 
context  in  which  the  term  is  embedded  and  by  which  the  term  can  be  interpreted 
does  not  exist.  Therefore,  in  this  report,  a  new  perspective  on  cognitive  task 
analysis  is  given.  A  framework  is  presented  in  which  important  views  on  cogni¬ 
tive  task  analysis  are  located.  This  framework  must  help  to  establish  a  mor<.. 
explicit  definition  of  what  is  meant  by  cognitive  task  analysis  and  must  give  a 
means  of  comparing  the  various  techniques  for  cognitive  task  analysis. 
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At  first  sight,  the  techniques  of  cognitive  task  analysis  seem  to  be  so  diverse  that 
no  common  characteristics  can  be  defined.  After  a  close  examination  of  the 
collected  techniques,  however,  three  main  views  can  be  identified.  These  views 
are; 

1  task  modelling 

2  knowledge  modelling 

3  cognitive  modelling. 

The  main  differences  among  the  three  views  can  be  characterised  as  follows.  The 
first  view,  the  task  modelling  view  takes  the  task  as  its  focus.  It  gives  an  answer 
to  the  question:  "what  goals  have  to  be  accomplished  by  performing  the  task?". 
The  second  view,  knowledge  modelling,  takes  the  task  requirements,  in  terms  of 
the  required  knowledge  and  the  required  strategies,  as  its  focus.  The  important 
questions  in  knowledge  modelling  are:  "which  performance  norms  are  given"  and 
"what  strategies  and  knowledge  should  be  used  in  order  to  accomplish  the 
goals?".  The  third  view,  cognitive  modelling,  takes  the  performer  of  the  task  as 
its  focus.  The  central  question  in  cognitive  modelling  is:  "what  performance  is 
displt^d  in  practice  in  order  to  accomplish  the  goals?".  It  gives  answers  to  the 
question;  "what  strategies  and  knowledge  are  actually  used  in  order  to  ac  com¬ 
plish  the  goals?". 

The  distinction  between  knowledge  modelling  and  cognitive  modelling  is  inspired 
by  Newell  (1982;  1992)  who  put  forth  several  ways  of  describing  problem  solving 
processes,  both  human  and  machine.  One  important  way  of  describing  the 
human  is  as  a  knowledge  system.  He  v  iewed  the  human  considered  as  a  knowl¬ 
edge-level  system  as  having  a  body  of  knowledge  and  a  set  of  goals,  so  that  it 
takes  actions  in  the  environment  that  iis  knowledge  indicates  will  attain  its  goals 
(cf.  Newell,  1982,  p.l02).  A  system  is  intelligent  to  the  degree  that  it  approxi¬ 
mates  a  knowledge-lewl  syrtem.  This  le'  el.  which  is  called  the  knowledge  level, 
is  comparable  to  the  knowledge  moaul'ing  view  mentioned  above.  There  is  a 
consensus  that  modelling  at  the  know!  dge  level  is  a  useful  intermediate  step  in 
the  development  of  an  expert  system  (Steels  &  McDermott,  1993).  Another 
important  way  of  describing  the  human  is  to  describe  the  way  a  knowledge-level 
system  is  realised  in  a  particular  mechanism.  This  level,  which  is  called  the 
symbol  lewl,  is  comparable  to  the  cognitive  modelling  view  mentioned  above. 
Physically  realizable  systems  can  at  best  only  approximate  a  knowledge-level 
system  and  thus  achieve  some  level  of  intelligence  that  is  less  than  perfect.  This 
is  because  of  the  computational  limits  upon  the  adaptive  powers  of  the  system. 

Both  the  knowledge  modelling  view  and  the  cognitive  modelling  view  can  be 
divided  again  into  two  different  branches;  one  that  is  oriented  towards  the 
analysis  of  procedural  knowledge;  another  that  is  oriented  towards  the  analysis 
of  declarative  knov,'ledge.  This  distinction  will  be  useful  in  comparing  different 
techniques  of  cognitive  task  analysis  with  each  other. 
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3.1  Task  Modelling 

The  task  modelling  view  takes  the  task  as  its  focus.  This  means  that  it  does  not 
consider  the  task  performer.  At  this  modelling  level,  it  must  even  be  possible  to 
describe  the  task  without  knowing  what  system  is  chosen  for  performing  the  task, 
a  human  task  performer  or  some  computer  system.  The  task  modeller  tries  to 
search  for  answers  conceding  the  goals  that  should  be  accomplished  by  the  task. 
Task  modelling  is  the  process  of  establishing  a  decomposition  of  the  task  into 
subtasks  and  the  process  of  establishing  the  context  of  the  task.  The  outcome  of 
this  process  is  a  task  model  which  contains  the  tasks  and  subtasks  which  have  to 
be  carried  out  in  order  to  accomplish  the  task’s  goal.  Information  concerning  the 
decomposition  of  the  task  very  often  can  be  found  in  manuals,  educational 
materials  and  procedures.  Another  possibility  of  information  gathering  is  inter¬ 
viewing  people  who  have  information  about  the  task  and  task  environment. 
These  people  are,  for  example,  persons  concerned  with  management  or  people 
who  are  responsible  for  training  and  education.  In  aiqf  case,  these  people  do  not 
necessarily  have  to  be  able  to  carry  out  the  task  themselves.  A  contmon  way  of 
representing  this  task  model  is  by  describing  it  in  the  form  of  a  tree  diagram.  An 
important  aspect  in  each  task  decomposition  is  to  determine  to  what  level  the 
task  must  be  decomposed.  This  stopping  criterion  should  preferably  be  deter¬ 
mined  on  the  basis  of  the  goal  of  the  cognitive  task  analysis  and  on  the  basis  of 
the  type  of  task  domain.  Traditional  task  analysis  has  become  less  popular  in 
circles  of  cognitive  task  analysis.  Some  researchers  in  this  area  however,  still 
keep  up  the  importance  of  task  analysis,  because  its  information  is  necessary  for 
specifying  task  performance  requirements  and  subsequently,  cognitive  require¬ 
ments  (Essens,  1991).  An  example  of  a  technique  that  can  be  considered  a  task 
modelling  technique  is  the  hierarchical  task  analysis  (Annett  &  Duncan,  1967; 
Sheperd,  1985).  Tliis  technique  will  be  described  later  on  in  this  report. 


3.2  Knowledge  Modelling 

The  knowledge  modelling  view'  takes  the  task  requirements  as  its  focus.  Task 
requirements  should  be  viewed  here  as  the  strategies  and  the  knowledge  the  task 
performer  must  use  in  order  to  accomplish  the  task’s  goal  in  an  optimal  way. 
The  important  questions  in  knowledge  modelling  are:  "what  performance  norms 
are  given"  and  "what  strategies  and  knowledge  are  necessary  in  order  to  accom¬ 
plish  the  goals  of  the  task?".  In  contrast  to  the  task  modelling  view,  the  kncwl- 
edge  modelling  view  certainly  takes  the  task  performer  into  account  in  the  task 
analysis.  However,  the  task  performer  is  not  regarded  as  a  regular  task  per¬ 
former,  but  he  or  she  is  viewed  as  a  genius  or  a  perfectly  intelligent  system,  that 
is  able  to  retries  all  required  knowledge  and  to  execute  the  required  strategies 
under  all  ciicuirislances.  A  task  performer  wiio  is  very  experienced  in  the 
execution  of  the  task  in  real  life  resembles  the  prototype  of  the  genius  most 
closely.  The  general  question  in  kncwledge  modelling  concerns  the  way  the  goals 
of  the  task  should  ideally  be  accomplished.  This  question  can  be  answered  by 
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investigating  the  two  branches  of  knowledge  modelling  that  ha\«  already  been 
mentioned;  knowledge  modelling  which  is  oriented  towards  procedural  knowl¬ 
edge  and  knowledge  modelling  which  is  oriented  towards  declarative  knowledge. 
Each  of  these  branches  is  described  separately  in  the  following  paragraphs.  One 
common  characteristic,  concerning  the  practical  usage  of  the  two  branches  of 
knowledge  modelling  is  the  time-consuming  process  of  applying  a  knowledge 
modelling  technique  to  a  task  domain  as  a  whole.  Therefore,  in  knowledge 
modelling,  one  subtask  of  the  task  is  very  often  selected  and  the  modelling 
technique  is  applied  to  this  subtask  alone.  The  selection  of  the  subtask  mostly 
depends  on  the  objective  of  knowledge  modelling.  Take  the  medical  domain  as 
an  example,  in  which  the  final  goal  of  knowledge  modelling  may  be  developing  a 
knowledge-based  system  that  should  support  the  pl^idan  with  diagnosis.  In  this 
case  the  subtask  of  "diagnosis"  is  selected  and  taken  for  further  knowledge 
modelling  investigation.  Subtasks  such  as  "performing  tests"  or  "prescribing 
medication"  are  not  selected,  because  these  subtasks,  in  this  example,  are  not 
supported  by  a  knowledge-based  system. 

CL  Knowledge  Modelling  pcusing  on  procedural  knowledge 

This  branch  of  knowledge  modelling  tries  to  give  answers  to  the  way  the  task’s 
goals  should  ideally  be  accomplished  by  focusing  on  the  most  optimal  procedural 
knowledge  that  is  required  to  carry  out  the  task.  Procedural  knowledge  is 
defined  here  as  mental  representations  consisting  of  an  ordered  series  of  steps 
for  accomplishing  a  taskls  goal.  Experts  are  regarded  as  important  information 
sources  in  knowledge  modelling,  because  they  are  the  ones  that  most  likely  take 
the  optimal  strategies  in  carrying  out  the  task.  Information  about  required 
strategies  or  sequences  of  steps  belongs  to  the  procedural  task  information  which 
often  carmot  be  found  on  paper.  Therefore  research  on  artificial  intelligence 
strives  for  identifying  and  registrating  generic  strategies  in  common  types  of 
problem-solving  tasks;  the  so-called  generic  tasks  (Chandrasekaran,  1983).  With 
the  aid  of  a  library  of  generic  tasks  the  identification  of  strategies  of  new  tasks, 
assumed  to  be  examples  of  generic  tasks,  can  be  speeded  up  (Schaafstal  & 
Schraagen,  1992). 

Generic  tasks  can  be  defined  us  general  categories  of  tasks,  which  group 
particular  tasks  together  because  of  their  corresponding  problem  solving  charac¬ 
teristics.  This  means,  for  instance,  that  tasks  which  belong  to  the  same  generic 
task  have  the  same  global  characteristics  of  the  problem,  the  same  global 
characteristics  of  the  solution  and  the  same  global  characteristics  of  the  process 
of  finding  one  or  more  solutions  (the  strategies).  The  generic  task  categories 
which  have  already  been  defined,  include  classification,  assessment,  diagnosis, 
design,  confi^ration  and  planning  (Schreiber,  Breuker  &  Wielinga,  1993). 
Identification  of  a  task  as  an  example  of  a  generic  task  takes  place  if  task 
characteristics  of  the  particular  task  at  hand  match  against  characteristics  of  a 
generic  task.  An  example  of  this  matching  process  can  be  given  by  the  identifica¬ 
tion  of  the  task  of  contracting  a  person  as  a  generic  assessment  task  or  the  task 
of  troubleshooting  in  a  technical  system  as  an  example  of  the  generic  diagnosis 
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ta<;k  Another  example  of  a  generic  task  is  hierarchical  classification,  that  uses 
the  "establish  and  refine"  method  and  assumes  the  existence  of  a  classification 
hierarchy.  The  underlying  idea  is  that  generic  tasks  can  be  combined  in  order  to 
solve  more  complex  tasks. 

b.  Knowledge  Modelling  focusing  un  declarative  knowledge 

This  branch  of  knowledge  modelling  specifically  focuses  on  declarative  knowl¬ 
edge  that  is  required  in  order  to  accomplish  the  task’s  goals.  Although  this  kind 
of  research  can  more  easily  profit  from  written  information  about  required 
knowledge,  in  this  modelling  view,  experienced  task  performers  are  of  great 
value  too;  especially  if  written  information  is  lacking.  Describing  knowledge  on 
the  level  of  the  knowledge  models,  means  that  the  knowledge  modeller  identifies 
the  views  that  can  be  put  upon  the  way  the  domain  concepts  are  related  and  the 
organisation  of  concepts  and  relations.  If  a  generic  task  category  has  been 
identified  for  a  certain  task,  the  views  with  which  the  task  domain  is  approached 
can  be  more  easily  identified  as  well.  Another  word  for  "perspective"  in  this 
context  is  "domain  model"  or  "knowledge  model".  Different  types  of  domain 
models  can  be  identified.  Examples  are:  structural,  behavioural,  functional, 
topological,  fault  models,  component  models,  etcetera.  Trying  to  match  the 
specific  task  of  troubleshooting,  as  mentioned  above,  one  could  identify  this  task 
as  a  diagnosis  task.  In  diagnosing  a  technical  system,  the  functional  model  and 
the  component  model,  for  instance,  lend  themselves  well  to  reason  about  the 
system  faults.  A  lot  of  progress  has  been  made  in  artificial  intelligence  in  this 
area  of  research  (Steels,  1992;  Benjamins,  1993). 


3.3  Cognitive  Modelling 

Cognitive  modelling  takes  the  performer  of  the  task  as  its  focus.  The  important 
questions  in  cognitive  modelling  are:  "what  performance  is  displayed  in  practice 
in  order  to  accomplish  the  task’s  goals"  and  "what  strategies  and  knowledge  are 
actually  used  in  order  to  accomplish  the  task’s  goals”.  In  contrast  with  knowledge 
modelling,  not  only  the  gernus  or  expert  task  performer  is  used  as  single  human 
information  resource  but  the  less  experienced  task  performer  or  even  the 
beginner  are  used  for  this  purpose  as  well.  However,  here  the  expert  is  not 
considered  as  a  perfectly  intelligent  system  but  as  a  task  performer  who  can  also 
make  mistakes  while  carrying  out  his  task  and  especially  these  mistakes  are 
investigated  thoroughly  in  cognitive  modelling.  To  clarify  the  difference  between 
knowledge  modelling  and  cognitive  modelling,  one  can  think  of  knowledge 
modelling  as  a  description  of  the  performance  norms  and  of  cognitive  modelling 
as  a  description  of  the  actual  performance  itself.  The  performance  norms  discuss 
the  waj'  the  task%  goals  should  ideally  be  accomplished,  whereas  the  description 
of  cognitive  perfonnance  discusses  me  way  the  task^  goals  are,  or  are  not, 
accomplished  in  practice.  Knowledge  modelling  views  performers  as  perfect 
systems  that  are  not  hampered  by  any  kind  of  limitation  of  information  process¬ 
ing.  Cognitive  modelling  views  human  performers  as  systems  endowed  with 
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limitations  on  their  information  processing,  for  instance  human  performers 
cannot  always  perform  tasks  perfectly  because  of  their  limited  workit^  memory. 
Cognitive  modelling  can  also  be  divided  into  two  branches  which  correspond  to 
the  branches  identified  in  knowledge  modelling:  cognitive  modelling  oriented 
towards  procedural  knowledge  and  cognitive  modelling  oriented  towards 
declarative  knowledge. 

a.  Cognitive  Modelling  focusing  on  procedural  knowledge 

This  branch  of  cognitive  modelling  turns  the  interest  to  the  study  of  procedural 
knowledge  that  are  used  in  practice  to  carry  out  a  task.  A  cognitive  modeller 
tries  to  analyse  the  various  strategies  that  are  actually  used  by  expert  and  less 
experienced  task  performers  in  carrying  out  their  task.  In  this  branch,  verbal 
protocol  analysis  is  often  used  as  a  method  for  obtaining  procedural  knowledge 
(Ericsson  &  Simon,  1984).  Protocol  analysis  is  a  technique  for  analysing  verbal 
protocols,  obtained  from  task  performers  thinking  aloud  while  carrying  out  their 
task.  Hence,  protocol  analysis  is  not  based  on  what  task  performers  themselves 
think  they  are  doing  while  carrying  out  their  task  but  rather  on  the  actual  verbal 
statements  that  are  taken  as  data.  These  data  are  used  for  testing  hypotheses  on 
the  presumed  procedural  knowledge  that  underly  the  observed  cognitive  behav¬ 
iour.  In  this  area  much  research  is  being  done  concerning  the  differences 
between  novices  and  experts  (Schraagen,  1993).  Human  limitations  are  taken 
into  account  in  producing  the  cognitive  model.  The  modeller  studies  the  human 
performer’s  way  of  "walking  through"  the  task  tree,  created  during  task  model- 
ling,  and  represents  those  strategies  as  a  kind  of  cognitive  path.  For  instance, 
Schraagen  (1993)  found  that  experts,  when  designing  experiments,  used  a 
structured  approach  to  solving  both  novel  and  routine  problems,  whereas  novices 
used  a  quasi-random  approach.  Experts  started  by  an  elaborate  problem  orienta¬ 
tion  phase,  whereas  novices  jumped  right  in  the  middle  of  the  problem.  In  the 
domain  of  diagnosis,  Schaafstal  (1993)  found  differences  in  procedural  knowl¬ 
edge  of  experts  and  novices  diagnosing  a  f:\ult  in  a  paper  mill.  In  this  domain  as 
well,  experts  have  a  much  more  structured  approach  to  diagnosis  than  novices, 
who  often  forgot  to  consider  the  seriousness  of  the  problem  or  to  evaluate  their 
solution.  An  example  of  a  technique  at  the  level  of  cognitive  modelling  is  the 
method  of  analysing  Goals,  Operators,  Methods  and  Selection  Rules  (GOMS) 
(Card,  Moran  &  Newell,  1983).  Tnis  technique  is  discussed  later  on  in  this 
report. 

h.  Cognitive  Modelling  focusing  on  declarative  knowledge 

This  branch  of  cognitive  modelling  is  concerned  with  the  analysis  of  declarative 
knowledge  which  are  used  in  practice  to  carry  out  a  task.  The  different  types  or 
models  of  knowledge  are  investigated  thoroughly  on  this  level  of  cognitive 
modelling;  it  is,  for  instance,  analysed  whether  a  task  performer  uses  a  causal 
model  to  solve  a  particular  problem,  instead  of  the  prescribed  functional  model. 
Various  knowledge  elicitation  techniques  may  be  employed  in  order  to  derive 
declarative  knowledge:  psychological  scaling  techniques  (Schvaneveldt,  Durso, 
Goldsmith,  Breen  &  Cooke,  1985;  Cooke  &  McDonald,  1987)  such  as  multi- 
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dimensional  scaling  (Kniskal  1964);  sorting  techniques  such  as,  hierarchical 
clustering  schemes  (Johnson,  1967)  and  multi-trial  free  recall  (Reitman  & 
Rueter,  1980);  eliciting  semantic  relations  for  networks  (Coote,  1993);  eliciting 
information  on  the  basis  of  "limited-information  tasks"  (Hoffiman,  1987);  inter¬ 
viewing  techniques,  which  make  use  of  domain-related  cues,  such  as  critical 
incidents  analysis  (Flanagan,  1954)  and  repertory  grid  analysis  (Boose,  1985)  and, 
to  a  lesser  extent,  think-aloud  protocols  and  retrospective  analysis  (Ericsson  & 
Simon,  1984). 


4  APPLYING  THE  MODELLING  VIEWS  IN  PRACTICE 

Now  that  the  relevant  aspects  of  the  three  modelling  views  are  described,  the 
practical  usage  of  each  modelling  view  will  be  elucidated  according  to  three 
prototypical  applications:  HTA  as  a  representative  of  task  modelling,  KADS  as  a 
representative  of  knowledge  modelling  and  GOMS  as  a  representative  of 
cognitive  modelling. 


4.1  Task  Modelling:  HTA 

Hierarchical  Task  Analysis  (HTA),  first  described  by  Annett  and  Duncan  (1967), 
is  a  formal  method  for  structuring  information  about  the  total  task  which  a 
human  has  to  perform.  HTA  is  a  form  of  task  analysis  which  tries  to  reveal  the 
goal-directed  behaviour  of  the  human.  The  central  concern  of  HTA  is  the 
decomposition  of  a  systems  task  into  operations  (or  subtasks)  linked  by  plans. 
The  fact  that  HTA  tries  to  decompose  a  human’s  task  into  operations,  makes  it 
an  example  of  a  task  modelling  technique.  The  aspect  of  linking  these  operations 
into  plans,  makes  it  also  a  candidate  for  a  knowledge  modelling  technique,  which 
is  concentrated  on  identifying  procedural  knowledge.  The  hierarchical  decompo¬ 
sition  of  the  task  is,  however,  emphasized  most  and  therefore  it  can  be  consid¬ 
ered  from  the  task  modelling  view  for  the  greater  part. 

In  decomposing  the  task,  HTA  tries  to  divide  the  operations  into  its  coratituent 
sub-operations,  as  long  as  the  sub-operations  are  mutually  exclusive  and  together 
form  the  higher-order  operation.  The  original  version  of  HTA  advised  general 
"stopping  rules"  to  apply  at  the  decomposition  process  but  the  current  version  is 
influenced  more  by  a  pragmatic  point  of  view  and  advises  the  task  analyst  to 
stop  the  decomposition  process  when  further  decomposing  would  have  no  profit 
to  the  task  analyst.  In  HTA  it  is  common  to  analyse  some  parts  of  the  task  in 
detail  (they  are  further  broken  down  in  operations)  and  other  parts  of  the  task 
only  superficially.  Tne  output  has  a  standard  format:  a  tree  diagram.  This  tree 
diagram  presents  the  operations  in  their  relative  positions  to  one  another. 
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Crawshaw^  Healey,  Hockey  and  Lambert  (1993)  used  HTA  as  one  of  the  sources 
of  information  to  analyse  the  task  of  a  watchkeeper  on  a  ship’s  bridge.  The  goal 
of  analysing  this  task  was  to  devise  scenarios  which  were  to  be  used  in  a  simula¬ 
tion  programme  of  the  task.  The  analysts  retrieved  information  for  the  HTA 
from  literature,  firom  their  own  observational  experiences  and  from  interviews 
with  numerous  seafarers.  A  part  of  the  tree  diagram  of  the  navigation  task  of  the 
watchkeeper  is  shown  in  Fig.  1.  This  figure  illustrates  the  decomposed  task  of 
the  watchkeeper  into  four  subtasks:  "avoid  grounding"  (3.2.1),  "avoid  collision" 
(3.2.2),  "avoid  adverse  weather  effects"  (3.2.3),  and  "control  ship  course  and 
speed"  (3.2.4).  The  figure  shows  that  the  subtask  of  avoid  collision  (3.2.2)  is 
further  analysed  and  broken  down  into  six  subtasks. 


Fig.  1  Example  of  an  outcome  of  an  HTA  for  the  navigation  task  of 
a  watchkeeper  on  a  ship’s  bridge. 


One  of  the  advantages  of  HTA  as  a  task  modelling  technique  is  that  it  gives  the 
task  analyst  a  clear  overview  of  the  context  of  the  task  and  of  its  requirements. 
In  cases  in  which  the  improvement  of  only  one  subtask  is  aimed  for,  it  makes 
more  sense  to  know  the  other  subtasks  which  are  necessary  to  accomplish  the 
task  goal  as  well.  One  could,  for  example,  easily  imagine  that  the  most  optimal 
change  in  the  process  of  accomplishing  a  task  goal  is  not  realised  by  the  im¬ 
provement  of  only  one  subtask  but  is  realised  by  a  reorganisation  of  other 
subtasks.  However,  in  practice  the  hierarchical  decomposition  is  not  always 
applied  to  the  total  number  of  operations  but  more  often  to  a  specific  part  of 
the  task.  One  of  the  disadvantages  of  HTA  is  that  the  validation  of  the  method 
is  difficult  because  different  people  might  decompose  a  task  in  different  ways.  So 
one  may  doubt  the  normative  character  of  the  task  description,  if  HTA  is  used 
as  a  task  modelling  technique.  Therefore  it  is  not  surprising  that  sub-operations 
can  be  devised  according  to  subjective  elements  or  that  sequences  of  operations 
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can  personally  be  determined.  It  seems,  the'-efore,  wise  to  have  multiple  analysts 
independently  carry  out  an  HTA,  and  resolve  disagreements  by  discussion.  Also, 
the  outcome  of  an  HTA  should  be  checked  thoroughly  with  experienced  task 
performers  or  educators  and  improved  upon  in  an  iterative  fashion. 


4.2  Knowledge  Modelling:  KADS 

Knowledge  modelling  that  is  oriented  both  towards  procedural  knowledge  as 
well  as  towards  declarative  knowledge  is  possible  within  one  kind  of  modelling 
technique.  An  example  of  such  a  modelling  technique  is  the  method  of  Knowl¬ 
edge  Analysis  and  Documentation  System  (KADS)  (Schreiber  et  al.,  1993). 
KADS  models  knowledge  in  terms  of  generic  tasks  and  in  terms  of  standard 
organizations  for  declarative  knowledge  and  is  used  for  the  development  of 
knowledge-based  systems.  The  importance  of  generic  tasks  in  KADS  is  that  it 
allows  the  system  developer  or  knowledge  modeller  to  make  use  of  established 
patterns  of  knowledge  use  (making  inferences)  and,  secondly,  to  use  generic 
tasks  in  interpreting  verbal  data.  This  technique  which  is  characterised  as  a  task 
oriented  approach  (Benjamins,  1993),  tries  to  find  out  which  performance  norms 
are  piescril^d  and  what  strategies  and  knowledge  should  be  used  in  order  to 
accomplish  the  task^  goals.  Task  oriented  approaches  try  to  model  problem 
solving  in  terms  of  the  knowledge  that  is  used  for  the  problem  solving.  This 
knowledge  is  organised  and  represented  by  generic  tasks  (e.g.  diagnosis,  repair, 
configuration). 

KADS  is  a  knowledge  modelling  technique  that  is  used  to  model  problem 
solving  by  identifying  tasks  at  various  levels  of  abstraction.  Despite  the  fact  that 
KADS  is  often  used  to  develop  knowledge-based  systems,  modelling  expertise  in 
KADS  is  done  independent  of  implementation  aspects.  The  outcome  of  KADS  is 
the  so-called  "model  of  expertise".  This  model  is  constituted  by  four  different 
types  of  knowledge:  domain  knowledge,  inference  knowledge,  task  knowledge 
and  strategic  knowledge.  These  different  types  of  knowledge  (in  KADS  terms: 
"layers")  are  used  to  gather  and  structure  the  knowledge.  Because  in  this  re¬ 
search  area  the  knowledge  gathered  on  problem  solving  comes  from  various 
information  sources  (written  materials  or  human  experts)  and  is  often  not 
complete,  KADS  uses  generic  tasks  (in  KADS  terms:  "interpretation  models")  in 
order  to  identify  the  particular  type  of  problem  solving.  If  a  particular  task  is 
matched  against  a  generic  task,  the  acquired  knowledge  can  more  easily  be 
interpreted  and  structured. 

The  underl3nng  idea  here  is  that  the  problem-solving  process  in  even  different 
domains  can  be  characterised  by  the  same  underlying  patterns  of  inference  and 
interpretation,  A  generic  task  or  interpretation  model  shows  which  reasoning 
steps  are  to  be  follovwd  to  solve  a  problem  in  a  given  type  of  problem.  From 
these  interpretation  models  one  can  derive  at  which  points  in  the  reasoning 
process  information  is  needed,  information  must  be  calculated,  solutions  are 
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generated  and  selected,  etcetera.  From  the  perspective  of  re-usability  and 
maintenance  a  library  of  generic  components  is  built  for  the  knowiedge  engineer. 
In  the  new  version  of  KADS  which  is  still  under  development,  named 
CommonKADS,  re-usable  structures  of  domain  models  (ontologies)  are  searched 
for  (Benjamins,  1993);  these  models  show  the  type  of  objects  within  a  given 
problem  domain  on  an  abstract  level  and  the  type  of  relations  that  must  exist 
between  those  objects.  This  ontological  knowledge  could,  for  example,  help 
students  in  training  troubleshooting  in  technical  devices  (teaching  system 
knowledge  on  a  functional  level). 

An  example  of  an  application  of  KADS  is  the  Sisyphus  project  in  which  KADS 
has  been  used  to  the  office  assignment  problem  (Schreiber  et  ai.,  1993).  This 
problem  can  be  described  as  allocating  rooms  to  employees,  while  satisfying 
numerous  constraints,  such  as  the  limited  number  of  rooms  (compared  with  the 
number  of  emplt^ees),  professional  peculiarities  and  personal  preferences.  The 
final  products  of  this  project  were  a  model  of  expertise  and  a  design  for  the 
application.  The  problem  solving  task  that  was  modelled  consisted  of  assigning 
rooms  to  emplc^ees.  Firstly,  a  description  of  the  major  entities  (employees, 
rooms,  projects)  and  relationships  (hierarchies,  project  assignments,  floor  plan) 
in  the  domain  was  created  to  gather  declarative  knowledge.  Secondly,  a  think- 
aloud  protocol  was  produced,  which  showed  how  an  expert  solves  a  particular 
office  assignment  problem,  to  gather  procedural  knowledge,  such  as  the  task 
structure  shown  in  Fig.  2  which  is  based  on  the  think-aloud  protocol.  It  shows 
how  particular  elements  (components,  plan,  etc.)  are  used  in  a  systematic  way  by 
particular  processes  (assemble,  select,  etc.). 


Fig.  2  Ex^ple  of  a  task  structure  created  by  using  KADS  techniques 
to  gather  inference  knowledge  about  the  office  assignment  problem. 


Thirdly,  the  protocol  was  used  to  form  some  idea  about  the  nature  of  the  task 
and  to  investigate  whether  it  resembled  some  known  (generic)  task.  Fourthly,  the 
office-assignment  task  was  classified  as  a  design  task;  although  the  solutions  are 
in  principle  enumerable  for  a  given  input  problem,  in  practice  the  solution  is  not 
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selected,  but  constructed  as  in  other  design  problems.  The  task  was  further 
specified  to  the  routine  task  of  allocation  and  the  corresponding  methods  and 
inferences  were  analysed.  On  the  basis  of  this  interpretation  model,  more 
knowledge  was  acquired  and  the  model  of  expertise  was  constructed.  Finally,  the 
behaviour  of  the  system  which  was  specified  in  this  model  of  expertise  was 
implemented. 

One  of  the  advantages  of  KADS  as  a  knowledge  modelling  technique  is  that  it 
makes  use  of  existing  models  of  problem  solving  in  acquiring  specific  task 
characteristics  and  task  requirements.  KADS  has  recognized  that  performance 
norms  are  lacking  very  often  or  are  written  down  only  partly.  Therefore  KADS 
strongly  turns  to  other  resources  of  information  to  acquire  procedural  knowledge 
and  declarative  knowledge.  If  written  information  is  lacking,  KADS  analyzes  the 
problem  solving  performance  of  experts  in  order  to  abstract  from  this 
performance  to  one  or  more  of  the  generic  models  of  problem  solving.  The 
generic  model  of  problem  solving  then  is  filled  in  with  task  specific  knowledge 
and  taken  as  a  kind  of  normative  performance  model.  One  of  the  disadvantages 
of  KADS  as  a  knowledge  modelling  technique  is  that  it  is  a  very  time  consuming 
process  to  execute  the  methodology.  KADS  is  primarily  developed  to  acquire 
knowledge  in  a  more  structured  way  in  order  to  develop  more  stable  and 
fundamental  knowledge*based  systems.  ITie  methodology  asks  from  the  analyst 
that  the  problem  solving  task  is  worked  out  in  such  a  detail  that  a  knowledge- 
based  system  can  be  designed  upon  this  outcome.  This  means  that  the  system 
must  be  able  to  function  as  an  intelligent  system  being  flexible  enough  to  reason 
with  different  kinds  of  knowledge  and  being  able  to  make  different  kinds  of 
inferences.  In  order  to  gain  better  insight  in  the  cognitive  performance  of  a 
human  task  performer,  it  is  not  necessary  to  model  completely  the  performance 
of  a  system  which  is  not  hampered  by  human  information  processing  limitations. 
KADS  focuses  loo  much  on  the  development  of  knowledge-based  systems  to  be 
directly  useable  for  cognitive  task  analysis. 


4.3  Cognitive  Modelling:  GOMS 

The  GOMS  method  (Card,  Moran  &  Newell.  1983)  is  a  technique  that  is  aimed 
at  modelling  the  human  performer.  The  acronym  GOMS  stands  for  Goals 
Operators  Methods  and  Selection  Rules.  GOMS  is  a  task  analysis  method  which 
tries  to  identify  the  methods,  operators  and  selection  rules  that  are  used  by  the 
human  task  performer  in  order  to  accomplish  his  or  her  goals.  This  means  that 
GOMS  focuses  on  the  knowledge  a  user  must  haw  about  the  methods,  operators 
and  selection  rules  or,  in  other  words,  the  "how-to-do-it"  knowledge.  One  could 
say  that  this  knowledge  is  the  user'^  representation  of  the  task.  Modelling  this 
"how-to-dc-it"  knowledge  corresponds  to  the  view  of  cognitive  modelling,  which 
tries  to  give  answers  to  the  question:  "what  strategies  and  knowledge  are  actually 
used  in  order  to  accomplish  the  task’s  goals?".  GOMS  explicitly  acknowledges 
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cognitive  limitations  on  task  performance,  such  as  working  memory  limitations 
and  deficiencies  in  knowledge  of  methods  for  carrying  out  tasks. 

The  goals,  operators,  methods  and  selection  rules  together  form  a  representation 
of  the  "how-to-do-it"  knowledge.  A  goal  is  a  symbolic  structure  that  defines  a 
state  of  affairs  to  be  achieved  and  determines  a  set  of  possible  methods  by  which 
it  may  be  accomplished.  Goals  and  subgoals  are  both  hierarchically  and 
sequentially  related  to  each  other,  so  that  the  complete  goal  structure  specifies 
the  conceptual  and  the  temporal  relationship  among  the  various  component 
tasks.  The  goal  structure  is  considered  the  plan  of  the  human  performer  for 
carrying  out  the  complete  task.  In  this  sense,  the  goal  structure  is  identical  to 
what  we  have  called  a  "task  structure".  Operations  are  defined  in  this  system  as 
mental  representations  of  the  various  elementary  functions  and  other  cognitive 
operations  that  are  performed  during  the  execution  of  the  task.  Method  are 
procedures  for  satisfying  specific  goals  and  subgoals.  Selection  rules  are  used  to 
specify  what  methods  should  be  used  to  satisfy  a  given  goal  or  subgoal  in  a 
specified  context.  When  several  methods  are  available  with  different 
characteristics  that  can  satisfy  the  same  goal,  selection  rules  are  used  to  pick  out 
the  method  that  is  appropriate  to  a  specific  context.  These  definitions  clearly 
indicate  that  GOMS  focuses  more  on  procedural  knowledge  than  on  declarative 
knowledge. 

As  an  example  of  a  GOMS  model,  imagine  the  user  goal  of  deleting  a  file  in  file 
manipulation  tasks  in  Macintosh  Finder.  In  order  to  accomplish  this  goal  a 
method  is  described  which  is  an  explicit  step-by-step  description  of  what  the  user 
has  to  do. 

Method  for  accomplishing  goal  of  deleting  a  file 
Step  1:  accomplish  goal  of  dragging  file  to  trash 
Step  2:  return  with  goal  accomplished 

Because  in  Macintosh  Finder  generalized  methods  can  be  found  in  order  to 
accomplish  various  goals,  a  general  submethod  corresponding  to  the  drag 
operation  is  defined. 

Method  for  accomplishing  goal  of  dragging  item  to  destination 
Step  1:  locate  item  for  item  on  screen 
Step  2:  move  cursor  to  item  icon  location 
Step  3:  hold  mouse  button  down 
Step  4:  locate  destination  icon  on  screen 
Step  5;  move  cursor  to  destination  icon 
Step  6:  verify  that  destination  icon  is  reverse-video 
Step  7:  release  mouse  button 
Step  8:  return  with  goal  accomplished 
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In  this  example,  goals,  methods  and  operators  are  described;  selection  rules, 
however,  are  not  inserted.  A  selection  rule  can  give  the  performer  the  choi(%  of, 
for  example,  taking  a  shortcut  of  operators,  to  take  an  alternative  sequence  of 
operations  etcetera.  These  terms  are  not  further  explained  here;  for  a  full 
description  see  Card  et  al.  (1983).  One  important  characteristic  of  a  GOMS 
model,  however,  still  has  to  be  mentioned.  It  is  the  fact  that  the  "how-to-do-it" 
knowledge  is  described  in  a  form  that  can  be  executed  by  a  human  being  or  a 
computer  program  (the  program  can  be  implemented  in  production  rules). 
Going  through  the  GOMS  description  the  actions  are  executed  and  the  task  is 
actually  carried  out.  The  goal  of  describing  the  knowledge  at  such  a  low  level  is 
to  derive  execution  times  and  predictions  of  learning  by  counting  the  number  of 
statements  or  production  rules. 

GOMS  may  be  viewed  upon  as  a  representative  of  cognitive  modelling  which  is 
one  of  few  methods  which  does  take  the  analysis  of  the  user’s  representation  of 
the  task  into  account  One  could,  however,  doubt  whether  GOMS  is  a  purely 
cognitive  modelling  method  in  the  above  sense  of  the  word.  In  order  to  be  able 
to  produce  a  user’s  representation  of  the  task,  the  job  task  and  its  environment 
are  represented.  Identifying  and  defining  the  user^  goals  is  said  to  be  dr^cult 
(Kieras,  1993),  because  the  analyst  must  examine  the  task  that  the  user  is  trying 
to  accomplish  in  such  detail  that  he  or  she  often  has  to  go  beyond  the  specific 
system  to  the  context  in  which  the  system  is  being  used.  This  seems  to 
correspond  to  our  definition  of  task  modelling.  Furthermore,  the  user’s  goals  are 
not  derived  from  the  user  directly  or  even  from  the  user  at  all  but  they  are  often 
just  speculated  upon  by  the  task  analyst.  In  situations  in  which  the  mjyority  of 
the  goals  is  device-dependent  they  are  relatively  easy  to  define.  Device¬ 
dependent  or  device-specific  goals  are  thin^  that  a  user  must  accomplish 
because  of  the  design  or  nature  of  the  device.  However,  in  situations  in  which 
natural  goals  are  important  too,  the  job  is  more  difficult  to  a  task  analyst 
because  these  goals  are  not  device-dependent  but  rather  user-dependent 
(depending  on  the  user’s  prior  experience  with  corresponding  systems  for 
instance).  In  order  to  reflect  the  user’s  concept  of  the  system,  the  anailyst  must 
attempt  to  identify  the  natural  goals  that  the  users  have.  This  process  can  better 
be  called  judgment  calling  than  cognitive  modelling.  An  example  of  a  judgment 
call  for  moving  text  in  the  text  processing  program  MacWrite,  is  that  the  user 
views  moving  text  as  first  cutting,  then  pasting,  rather  than  as  a  single  unitary 
move  operation.  So  at  the  same  time  this  feature  of  GOMS  is  one  of  the  weak 
aspects  of  the  method,  because  from  a  cognitive  modelling  point  of  view  it  is  not 
appropriate  for  the  analyst  to  make  assumptions  about  how  humans  view  a  task 
or  a  system.  This  critique  is  countered  by  GOMS  adherents  who  argue  that  it  is 
not  practical  to  collect  data  on  how  users  view  and  decompose  tasks. 

Tnerefore,  it  is  suggested  here  that  the  theoretical  basis  of  the  GOMS  method 
can  be  seen  as  contributing  to  the  cognitive  modelling  view  but  the  practical 
usage  of  GOMS  makes  it  belong  to  a  knowledge  modelling  technique.  Another 
practical  problem,  which  makes  the  cognitive  modelling  view  of  GOMS  doubtful. 
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is  the  fact  that  mai^  cognitive  processes  are  tou  difficult  to  analyze  in  a  practical 
context.  This  problem  is  even  agreed  upon  by  GOMS  adherents.  Examples  of 
such  cognitive  processes  are  reading,  problem-solving,  debugging,  and  so  forth. 
The  solution  which  the  GOMS  method  offers  to  this  problem  is  to  simply  bypass 
a  complex  process  by  representing  it  with  a  “dumiity"  or  "placeholder"  operator. 
Representing  a  bypassed  process  consists  of  using  an  analyst-defined  operator 
together  with  information  in  the  task  description  as  place  holders  to  document 
that  the  process  is  taking  place  and  what  its  assumed  results  are.  In  other  words, 
GOMS  can  produce  not  mar^  cognitive  modelling  results  in  tasks  which  consist 
of  lots  of  complex  processes.  It  is  not  surprising  that  most  of  the  GOMS 
applications  are  restricted  to  modelling  micro-  or  low-level  processes  which 
demand  little  efifort  from  human  cognition.  Therefore  GOMS  has  mostly  been 
used  for  analysing  expert  behaviour  ir  routine-like  tasks. 

An  example  of  an  application  of  GOMS  is  the  Ernestine  project  (Gray,  John  & 
Atwood,  1952,  1993).  In  this  project,  GOMS  was  used  to  test  its  applicability  to 
real-world  design  problems.  GOMS  models  were  defined  here  to  predict  the 
performance  of  the  toll  and  assistance  operator  (TAO)  using  a  new  workstation. 
The  TAO  is  the  operator  you  get  when  you  dial  0.  Their  job  is  to  assist  the 
customer  in  completing  calls  and  to  record  the  correct  billing.  Among  other 
tasks,  TAOs  handle  person-to-person  calls,  collect  calls,  calling-card  calls,  and 
calls  billed  to  a  third  number.  The  goal  of  using  GOMS  was  to  evaluate  two 
TAO  workstations — a  current  workstation  and  a  proposed  workstation. 

With  the  help  of  Operator  Services  Personnel  relevant  and  common  procedures 
executed  by  the  TAOs  were  depicted  and  used  as  benchmarks  for  modelling.  To 
model  these  benchmarks  two  different  approaches  were  used:  observation-based 
models  of  performers  interacting  with  the  current  workstation  and  models  based 
on  the  specifications  of  the  proposed  workstation.  Because  the  TAO’s  task  is 
characterised  by  doing  several  things  in  parallel  when  processing  a  customer’s 
request  CPM-GOMS  was  used  to  display  these  parallel  activities  and  to  calculate 
total  task  times  (John,  1988). 

CPM-GOMS  is  an  extension  to  GOMS  which  distinguishes  among  Cognition, 
Perception  and  Motor  operators  and  uses  the  Critical  Path  Method  (developed 
for  project  management).  The  critical  path  is  an  important  concept  in  analysing 
the  total  task  time  for  complex  parallel  tasks.  When  activities  occur  in  parallel 
one  sequence  of  activities  will  take  more  tiHie  than  parallel  sequences  of 
activities;  the  critical  path  is  the  sequence  of  activities  which  takes  the  longest 
and  determines  the  total  time  for  the  entire  task.  In  CPM-GOMS  the  parallelism 
of  the  task  is  represented  in  a  schedule  chart  (see  Fig.  3). 
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Fig.  3  Example  of  a  section  of  a  schedule  chart  created  by  using 
CPM-GOMS  techniques  to  evaluate  two  different  workstations.  From 
this  part  of  the  call  the  proposed  workstation  (bottom)  has  removed 
two  keystrokes  which  had  required  seven  motor  and  three  cognitive 
operators.  However  none  of  the  ten  operators  removed  were  along 
the  critical  path  (showTi  in  bold). 


In  this  chart  each  activity  in  handling  a  call  is  represented  as  a  box  with  an 
associated  duration.  Dependencies  between  activities  are  represented  as  lines 
connecting  the  boxes.  The  critical  path  is  also  displayed  in  this  chart  (bold  lines). 
The  boxes  and  their  dependency  lines  are  drawn  according  to  a  detailed 
understanding  of  the  TAO’s  task  and  goal  decomposition  (John,  1990). 

As  a  result  of  the  analysis,  the  CPM-GOMS  models  predict  that  the  proposed 
workstation  will  be  3%  slower  than  the  current  workstation.  These  results  were 
consistent  with  the  empirical  data  obtained  later  on  fix)m  a  field  trial.  The 
reason  wl^  the  proposed  workstation  was  slower  than  the  current  workstation 
was  explained  by  CPM-GOMS  as  follows.  Although  the  proposed  workstation 
generally  had  fewer  keystrokes  than  the  current  workstation,  the  new  procedure 
put  more  keystrokes  on  the  critical  path,  especially  to  the  end  of  the  call,  rather 
than  in  the  slack  time,  increasing  the  length  of  the  call.  In  addition,  in  the 
proposed  keyboard  the  close  spacing  of  the  keys  encouraged  the  right  hand  to 
press  all  keys;  CPM-GOMS  predicted  that  this  would  be  slower  than  the  old 
procedure  of  using  the  left  hand  for  certain  keys.  These  two  examples  make 
clear  that  CPM-GOMS  correctly  predicted  the  proposed  workstation  to  be 
slower,  while  the  manufacturer  predicted  that  the  proposed  workstation  would  be 
faster  than  the  current  workstation.  Based  on  the  GOMS  task  analysis,  the 
phone  compai^  had  to  reconsider  the  investment  of  the  proposed  workstation. 

One  of  the  advantages  of  GOMS  as  a  cognitive  modelling  technique  is  that  it 
allows  to  predict  performance  in  human-computer  interactions  very  accurately.  If 
human-computer  interaction  tasks  make  an  appeal  to  micro-  or  low-level 
processes  for  the  greater  part,  then  GOMS  is  a  very  good  predictor  for  human 
performance.  However,  at  the  same  time  this  advantage  could  be  seen  as  a 
disadvantage,  because  GOMS  is  not  good  at  predicting  human  performance  if 
tasks  are  not  routine-like  but  have  a  more  problem  solving  character  and  make 
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an  appeal  to  higher,  cognitive,  processes  problem  solving  tasks.  For  these  kind  of 
cognitive  and  t^mamic  tasks  GOMS  is  not  suitable. 


5  INmEGRATION  OF  IHE  MODELUNG  PERSPECTIVES 

Now  that  the  three  modelling  views  are  described  according  to  their  prototypical 
applications,  this  chapter  turns  to  the  integration  of  those  views  into  cognitive 
task  analysis.  Firstly,  the  tugumentation  behind  the  integration  of  cognitive  task 
analysis  is  given.  Secondly,  an  example  of  cognitive  task  analysis  as  it  could  be 
used  in  practice  is  given. 


5.1  Cognitive  Task  Analysis 

Cognitive  task  analysis  (CTA),  as  we  define  it,  is  a  form  of  analysing  a  task 
which  consists  of  the  three  views  together:  task  modelling,  knowledge  modelling 
and  cognitive  modelling.  If  only  one  of  these  views  is  used,  it  cannot  be  called  a 
cognitive  task  analysis.  An  analysts,  for  instance,  which  only  contaii.s  cognitive 
m^eiling  does  not  gather  enough  information  about  the  tas’c,  the  task 
environment  and  the  task  requirements.  TTie  same  can  be  said  for  the  other  two 
views  if  they  are  used  in  separation.  Because  a  cognitive  task  analysis  must 
minimally  be  able  to  gather  information  about  the  task,  the  task  environment, 
the  task  requirements  and  the  human  performance  of  the  task,  all  three 
modelling  views  are  important. 

The  order  in  which  these  three  forms  of  modelling  can  best  be  used  is  an  order 
in  which  task  modelling  is  the  first  modelling  activity,  knowledge  modelling  the 
second  and  cognitive  modelling  the  third  activity  This  order  is  argued  for 
because  analysing  and  interpreting  human  task  performance  (cognitive 
modelling)  is  hardly  possible  without  knowing  the  task,  the  subtasks,  the  task 
environment  (task  modelling)  or  without  knowing  the  requirements  of  the  task 
(knowledge  modelling).  Wiihou;:  knowledge  modelling,  it  is  also  difficult  to 
imagine  what  interpretation  can  be  given  to  the  output  of  cognitive  modelling.  In 
the  literature  the  term  cognitive  task  analysis  is  often  misplaced:  in  situations  in 
which  in  fact  only  cognitive  modelling  has  taken  place,  researchers  do  refer  to 
the  method  as  cognitive  task  analysis. 

However,  although  the  framework  for  the  ana’.ysio  of — and  examination  of — 
cognitive  tasks  helps  to  establish  a  more  explicit  definition  of  what  is  meant  by 
cognitiw  task  analysis,  it  <auses  practical  problems  at  the  same  time.  A 
consequence  of  defining  cognitive  task  analysis  as  do  could  have  some 
practical  problems,  because  the  piucess  of  modelling  the  task,  the  knowledge 
and  the  human  cognition  could  ask  more  time  than  was  expected  before. 
Therefore,  we  advise  to  coraider  th?  final  goal  of  the  analysis  in  practice 
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thoroughly,  before  using  various  modelling  techniques  in  order  to  gather 
information  about  the  task.  If  one  has  to  know  more  about  the  task,  the 
decomposition  of  the  task  into  suotasks  or  about  the  task  environment  one  can 
ojnfine  oneself  to  task  modelling.  In  this  case,  however,  one  cannot  call  this  kind 
of  modelling  a  cognitiw  task  analysis.  If  one  wants  to  know,  however,  which 
strategies  and  which  knowledge  are  necessary  to  carry  out  the  task  or  subtask, 
the  £q}piication  of  knowledge  modelling  is  required  after  a  task  modelling  has 
been  applied.  Again  one  cannot  call  this  kind  of  modelling  a  cognitive  task 
analysis.  Only  if  one  wants  to  know  whether  the  prescribed  strategies  and 
knowledge  are  actually  used  by  a  human  task  performer,  it  is  ne<»ssary  to  finish 
the  two  preceding  steps  with  cognitive  modelling.  In  this  case  one  can  call  the 
analysis  a  cognitive  task  analysis.  It  is  possible  that  cognitive  modelling  is  used  by 
itself  and  that  its  results  are  sufficient  to  the  goal  of  the  analysis;  in  these  cases, 
however,  we  prefer  to  call  this  kind  of  analysis,  in  which  the  task  is  not  studied 
in  a  detailed  way,  cognitive  modelling  or  cognitive  analysis. 

Summing  up,  cognitive  task  analysis  can  be  viewed  from  three  main  views:  the 
task  modelling  view,  the  knowledge  modelling  view  and  the  cognitive  modelling 
view.  In  contrast  to  the  latter  two  views,  a  task  modelling  view  emphasizes  the 
task  itself  and  does  not  consider  the  task  performer.  Task  modelling  is  the 
process  of  establishing  a  decomposition  of  the  task  into  subtasks  and  the  process 
of  establishing  the  context  of  the  task.  The  other  two  views,  knowledge 
modelling  and  cognitive  modelling,  do,  however,  often  start  with  the  task 
performer.  Intelligent  systems,  such  as  human  task  performers,  can  be  described 
at  either  of  these  levels.  A  knowledge  level  description  emphasizes  the 
knowledge  contents  of  a  system  (e.g.,  goals,  actions  and  knowledge  used  in  a 
rational  way),  whereas  the  symbol  level  describes  its  computational  realization. 
This  means  that  knowledge  modelling  tries  to  find  out  which  performance  norms 
are  prescribed  and  what  strategies  and  knowledge  should  be  used  in  order  to 
accomplish  the  task’s  goals,  whereas  cognitive  modelling  tries  to  find  out  what 
performance  is  actually  displayed  in  practice  in  order  to  accomplish  the  goals. 


52  Cognitive  Task  Analysis  in  Practice 

A  complete  application  of  cognitive  task  analysis,  as  it  is  meant  by  the 
framework,  cannot  be  found.  Therefore  an  example  of  an  integrated  version  of 
cognitive  task  analysis  is  described  here.  As  an  example  of  a  domain  to  which 
the  cognitive  task  analysis  could  be  applied,  the  orthopaedic  shoe  technician^ 
task  is  chosen.  The  orthopaedic  shoe  technician’s  task  is  meant  to  provide 
orthopaedic  shoeware  for  deformed  feet.  The  final  goal  of  the  cognitiw  task 
analysis  is  to  design  a  tutoring  system  for  junior  onhopaedic  shoe  technicians, 

TTie  first  modelling  view  applied  was  the  task  modelling  view.  The  main  question 
to  be  answered  in  this  phase  was  ’Svhai  goals  have  to  be  accomplished"  to 
provide  shoeware.  The  outcome  of  this  modelling  view  was  two-fold:  a  task 
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model  of  the  decomposed  task  and  insight  in  the  task  environment  The  method 
followed  to  reach  these  task  modelling  outcomes  consisted  of  interviewing 
subject-matter  experts,  such  as  practising  orthopaedic  shoe  technicians, 
orthopaedic  shoe  technicians  functioning  in  organisational  and  management 
positions  and  rehabilitation  physicians  and  by  observing  them  performing  their 
tasks. 


Fig.  4  Example  of  a  task  model  of  the  decomposed  task  of  providing 
orthopaedic  shoeware. 


The  first  outcome,  the  task  model  of  the  decomposed  task  is  partly  illustrated  in 
Fig.  4.  The  main  task  of  providing  shoeware  can  be  decomposed  into  the 
following  general  subtasks:  gathering  information,  testing,  designing,  fitting, 
producing,  finishing  and  checking.  These  subiasks  can  be  decomposed  again  into 
the  following  subtasks:  gathering  information  from  the  physicians,  gathering 
information  from  the  clients  themselves,  observing  the  clients'  movements, 
measuring  the  clients’  relevant  foot  parameters,  designing  the  first  shoe-concepts, 
designing  shoe-lasts,  fitting  the  clients’  test-shoes,  retouching  the  test-shoes, 
producing  the  shoe^  soles,  producing  the  shoe^  legs,  delivering  the  final  shoe- 
products,  refining  the  products  (in  case  of  few  complaints,  otherwise  the 
preceding  activities  must  be  iterated),  checking  the  clients’  feet  and  the  clients’ 
shoes  after  some  time.  The  task  model  is  important  for  identifying  problematic 
task  elements  and  for  gaining  insight  into  the  task  domain. 
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The  second  outcome  of  the  task  modelling  was  to  obtain  information  about  the 
task  environment.  This  information  was  obtained  by  confronting  the  subject- 
matter  experts  with  the  produced  task  model.  They  came  up  with  exceptions  in 
task  execution  which  were  influenced  by  the  various  situations  under  which  the 
task  could  be  executed;  in  doing  so  they  gave  the  task  analysts  more  insight  in 
the  context  of  the  task  and  were  able  to  indicate  the  problematic  issues  in 
orthopaedic  shoe  technic.  The  situations  that  make  the  task  performer  deviate 
from  the  task  model,  are  situations  in  which  a  number  of  subtasks  are  skipped. 
This  deviated  task  execution  can  be  found  in  situations  in  which  shoeware  is 
provided  to  already  known  clients,  in  contrast  to  situations  in  which  shoeware  is 
provided  to  novel  clients,  the  so-called  first  shoe  provision.  In  providing 
shoeware  to  already  known  client?  a  number  of  subtasks  can  be  skipped,  because 
most  information  of  the  client  is  already  gathered  and  relatively  few  changes 
have  occurred  since  the  first  shoe  provision.  In  further  investigating  this 
difference,  it  turned  out  that  most  problems  occurred  with  first  shoe  provisions 
and  that  these  problems  often  were  restricted  to  the  first  three  subtasks,  which 
together  form  the  "measurement  taking  process":  information  gathering,  testing 
and  designing.  In  the  measurement  taking  process  the  shoe  technician  notes  the 
client’s  problems,  takes  measurements  of  the  feet  and  designs  an  initial  shoe 
provision 

The  second  modelling  view,  knowledge  modelling,  was  focusing  on  the  output  of 
the  task  modelling,  in  this  case  the  three  problematic  subtasks  of  providing 
orthopaedic  shoeware:  information  gathering,  testing  and  designing.  The  relevant 
questions  in  this  phase  are  "which  performance  norms  are  prescribed"  and  "what 
strategies  and  knowledge  should  be  used  in  order  to  accomplish  the  goals  of 
providing  shoeware".  In  viewing  these  activities  from  a  knowledge  modelling 
point  of  view,  in  the  first  place  procedural  knowledge  were  to  be  identified. 
These  procedural  knowledge  would  show  how  expert  shoe  technicians 
approached  the  measurement  taking  process,  and  how  they  divided  this  process 
into  an  ordered  set  of  subtasks.  In  order  to  be  able  to  identify  the  task  of 
providing  orthopaedic  shoeware  as  some  kind  of  generic  task  more  information 
was  gathered  about  the  task  domain  by  interviewing  subject-matter  experts,  such 
as  orthopaedic  shoe  technicians,  rehabilitation  physicians  and  orthopaedic 
surgeons  and  by  observing  them  in  carrying  out  their  jobs,  while  being  asked  to 
think  aloud.  A  global  identification  of  two  generic  tasks  was  made  first:  diagnosis 
and  design.  As  the  outcome  of  the  knowledge  modelling  phase  a  normative 
cognitive  performance  model  was  made  on  the  basis  of  a  limited  number  of 
verbal  protocols  and  on  the  basis  of  the  two  generic  tasks.  The  result  was  a 
merging  of  the  diagnosis  and  the  design  task  applied  to  domain-specific  concepts. 
The  result  of  this  merging  was  a  normative  model  or  a  coding  scheme  for  verbal 
protocols  to  be  obtained  in  the  cognitive  modelling  phase. 

For  instance,  one  category  of  the  coding  scheme,  which  originates  from  the 
generic  task  of  diagnosis,  was  "formulate  hypotheses".  This  category  refers  to 
activities  dealing  with  formulating  hypotheses  concerning  the  reduction  of  foot 
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functions,  concerning  the  reduction  of  foot  functionality  and  concerning  the 
social  limitations  to  the  shoe  client.  A  statement  in  a  verbal  protocol  which 
refers  to  the  category  of  formulating  hypotheses  concerning  reduction  of  foot 
functions  is:  "if  you  see  those  pressure  stains  on  the  feet,  you  would  expect  that 
the  client  should  have  returned  sooner  (to  the  orthopaedic  shoe  technician)". 
The  next  category  in  the  coding  scheme  was  to  test  those  hypotheses.  In  the 
normative  model  these  categories  of  cognitive  tasks,  among  others,  must  be 
carried  out  in  order  to  provide  a  final  shoe  product. 

In  investigating  the  normative  cognitive  performance  model  of  the  tasks  more 
thoroughly,  the  problems  the  junior  orthopaedic  shoe  technicians  encountered, 
were  expected  to  be  caused  among  other  things  by  a  lack  of  diagnostic 
knowledge.  The  assumption  was  that  more  experienced  shoe  technicians  already 
had  acquired  this  knowledge  over  the  years,  because  they  cooperated  intensively 
with  rehabilitation  plysicians  and  orthopaedic  surgeons  and  that  they  had 
implicitly  merged  diagnosis  and  design  knowledge  in  such  a  way  that  they  were 
not  aware  they  used  this  knowledge  to  deliver  optimal  orthopaedic  shoeware. 
However,  junior  shoe  technicians  are  also  expected  to  know  what  kind  of 
activities  must  be  performed  on  the  basis  of  the  specific  deformed  feet  of  the 
client  in  order  to  deliver  optimal  orthopaedic  shoeware. 

The  third  modelling  view,  cognitive  modelling,  was  focusing  too  on  the  three 
problematic  subtasks  of  shoe  provision:  information  gathering,  testing  and 
designing.  The  central  question  here  was  "what  performance  is  displayed  in 
practice  by  orthopaedic  shoe  technicians  and  especially  by  junior  orthopaedic 
shoe  technicians"  in  order  to  provide  shoeware.  The  outcome  of  this  modelling 
view  was  to  produce  performance  models  of  these  task  performers  and  to 
compare  these  descriptive  models  with  the  normative  performance  model, 
produced  in  the  knowledge  modelling  phase.  This  outcome  was  obtained  by 
observing  and  analysing  the  task  performers  carrying  out  their  tasks,  while  being 
asked  to  think  aloud. 

Three  conditions  of  executing  the  task  were  investigated  and  compared  with 
each  other  to  obtain  possibly  different  performance  models.  The  first  condition 
consisted  of  a  situation  in  which  the  task  was  executed  by  an  experienced 
orthopaedic  shoe  technician  in  cooperation  with  a  rehabilitation  physician.  The 
second  condition  consisted  of  a  situation  in  which  the  task  was  executed  by  an 
experienced  orthopaedic  shoe  technician  on  his  own.  The  third  condition 
consisted  of  a  situation  in  which  the  task  was  executed  by  a  junior  orthopaedic 
shoe  technician  on  his  own.  In  these  conditions  the  task  and  the  kind  of  foot 
deformation  do  not  vary,  only  the  task  performers  do  by  varying  their  type  of 
expertise  and  by  varying  their  level  of  expertise.  In  this  way  conclusions  can  be 
made  about  the  differences  in  task  execution  found  in  the  three  different 
conditions.  This  gives  us  further  insight  in  expert-novice  differences  and  in  the 
importance  of  collaborating  with  physicians.  Knowing  the  problems  junior 
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orthopaedic  shoe  technicians  encounter,  learning  material  can  be  constructed  to 
develop  a  tutoring  system. 


6  GENERAL  DISCUSSION 

A  framevrork  for  the  analysis  of — and  examination  of — cognitive  tasks  has  been 
presented.  This  firamework  has  been  created,  because  an  explicit  definition  of 
cognitive  task  analysis  does  not  exist  and  because  areas  of  research  should  be 
explored  in  the  field  of  cognitive  task  analysis.  Despite  the  fact  that  ma^ 
techniques  in  the  literature  are  labelled  "cogniti^^  task  analysis”,  they  show  little 
or  no  resemblance  with  each  other.  TTiis  diversity  is  caused  partly  by  the  fact 
that  each  technique  is  developed  to  accomplish  a  speciflc  goal. 

Generally  the  following  goals  of  cognitive  task  analysis  can  be  found: 
a  Generating  interface  specifications 
b  Assessing  and  improving  workload 

c  Describing  the  functional  structure  of  a  task  domain  for  function  allocation 
d  Designing  training  materials  and  generating  training  issues 
e  Diagnosing  performance  and  predicting  performance  time 
f  Developing  knowledge-based  systems. 

In  order  to  apply  cognitive  task  analysis  in  practice,  it  is  necessary  to  be  explicit 
about  the  way  the  goals  of  the  cognitive  task  analysis  can  be  accomplished. 
However,  we  have  already  mentioned  that  defining  cognitive  task  analysis  as 
modelling  the  task,  the  knowledge  and  the  human  cognition  is  a  time-consuming 
process.  Therefore  the  modelling  views  can  only  be  worked  out  with  equsd 
strength  in  every  application  of  cognitive  task  analysis  at  high  costs;  not  all  views 
are  equally  important  in  accomplishing  the  goal  of  each  analysis.  This  means  that 
the  contribution  of  each  of  the  modelling  views  to  cognitive  task  analysis  can 
vary.  The  framework  for  cognitive  task  analysis  gives  no  practical  guidelines 
about  the  weight  of  each  modelling  view  in  cognitive  task  analysis.  The  process 
of  determining  what  contribution  each  modelling  view  must  have  and  what 
analysis  technique  must  be  selected  strongly  depends  on  the  goals  of  the  analysis. 
These  issues  can  be  especially  interesting  for  applied  research.  Table  I  illustrates 
a  possible  relation  between  the  goals  of  cognitive  task  analysis  and  the  three 
modelling  views.  This  relation  and  the  relation  between  modelling  views  and 
analysis  could  be  investigated  further  in  the  future. 
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Table  I  Proposed  relative  weight  (low,  medium,  high)  of  the 
modelling  views  for  the  various  goals  of  cognitive  task  analysis. 


modelling  perspectives  J 

goals  of  cognitive 
task  analysis 

task  modelling 

knowledge  modelling 

cognitive  modelling  | 

generate  interface 
specifications 

improve 

workload 

functional 

structure 

designing  training 
material 

predicting 

preformance 

developing 

knowledge-based  systems 

-  low 

i/ZZ/'A  =>  medium 


Another  interesting  issue  for  applied  research  is  to  investigate  »he  order  in  which 
the  three  modelling  views  in  cognitive  task  analysis  could  be  used  best  and  to 
inx^stigate  what  kinds  of  modelling  views  and  techniques  can  be  applied  best  to 
analyse  particular  tasks  and  task  environments.  From  this  point  of  view  various 
dimensions  (e.g,  the  degree  of  structuredness  of  a  task)  could  be  created  to 
characterize  each  task  and  to  select  a  modelling  view  or  even  an  analysis 
technique  on  the  basis  of  the  identified  task  characteristics.  This  pre-analysis 
phase  could  be  carried  out  before  the  actual  cognitive  task  analysis  begins.  It 
would  be  convenient  to  check  a  couple  of  standard  items  against  a  given  task 
and  to  decide,  on  the  basis  of  this  result,  what  kind  of  modelling  view  or 
modelling  technique  can  best  be  used.  An  important  goal  of  doing  this  kind  of 
applied  scientific  research  is  to  develop  practical  guidelines  for  selecting 
appropriate  analysis  techniques  on  the  basis  of  the  goals  of  the  analysis  and  on 
the  basis  of  identified  task  characteristics. 

Fundamental  research  could  focus  more  on  the  bridging  of  the  gap  between  one 
sight  of  the  camp,  the  task  modelling  view,  and  the  other  sight  of  the  camp,  the 
cognitive  modelling  view.  It  is  interesting  to  invent  a  way  in  which  task 
requirements  could  be  acquired  in  such  a  way  that  they,  together  with  the  task’s 
goals,  can  easily  be  used  to  make  predictions  about  cogtiitive  performance.  The 
idea  of  focusing  not  only  on  the  task  modelling  and  cognitive  modelling  view  but 
on  the  knowledge  modelling  view  as  well,  in  which  task  requirements  and 
performance  norms  are  important,  is  shared  with  Essens,  Fallesen,  McCann, 
Cannon-Bowers  and  Dorfel  (1994).  Within  the  context  of  our  recent  research  of 
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decision  making  in  teams  the  acquisition  of  task  requirements  and  performance 
norms  also  plays  an  important  role  in  allocating  tasks  and  subtasks  to  specific 
functions  of  the  team.  The  assessment  of  cognitive  performance  of  the  team 
members  should  preferably  be  made  before  having  analyzed  the  (cognitive) 
performance  of  the  members  in  practice,  because  this  often  can  be  done  only  at 
high  costs.  Members  of  teams  are  cooperating  with  each  otlier  to  accomplish  one 
main  goal.  In  order  to  be  able  to  assess  the  cognitive  performance  of  the 
members  of  the  team  or  of  the  decision  maker  the  main  goal  must  be 
decomposed  in  various  subgoals  and  their  task  requirements  must  be  obtained.  If 
techniques  are  found  to  obtain  task  requirements  efficiently,  further  research 
must  focus  on  the  prediction  of  cognitive  performance  of  team  members  out  of 
the  subgoals  and  the  requirements  of  the  task. 
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APPENDIX 

In  order  to  gain  a  complete  view  on  the  relevant  literature,  a  literature  search 
was  carried  out  within  the  following  databases: 

•  NnS,  1964-1993/Sep  B2,  keywords:  cognitive  processes,  task  analysis  and  the 
combiiiation  of  these  two  keywords; 

•  SOCIAL  SaSEARCH,  1972-1993/Aug  Wl,  keywords:  task  analysis, 
cognitive?  and  the  combination  of  these  two  keywords; 

•  TDCK,  keywords:  cogniti*  and  the  Dutch  word  for  task  analysis 
("taakanalyse")  or  task  adj  analysis; 

•  PsydNFO,  1967-1993/Sep,  keyw>rds:  task  analysis,  cognitive?  and  a 
combination  of  these  two  keywords;  models  or  methodology  or  measurement 
or  methods  and  a  combination  of  the  three  keywords; 

The  result  of  this  literature  search  was  a  total  amount  of  82  references.  Most  of 
these  references  were  not  appropriate  to  be  pursued,  because  they  were  either 
too  old  (published  before  1985),  too  vague  or  too  specific  (e.g.  educational 
analyses).  From  the  other  references  the  following  techniques  or  methodologies 
were  considered  to  be  appropriate  to  be  studied  in  the  perspective  of  this  report: 

Analysis  for  Task  Object  Modelling  (ATOM)  (Walsh;  in  Diaper,  1989) 

Contextual  Module  Analysis  (CMA)  (Tennyson,  Elmore  &  Snyder,  1992) 
Cognitive  Reliability  Analysis  (CRA)  (Hollnagel,  1991) 

Cognitive  Tasks;  A  Meta-Analysis  of  Methods  (Aim,  1992) 

Cognitive  Task  Analysis  (CTA)  (Bainbridge,  1989) 

Cognitive  Task  Analysis  (CTA)  (Klein,  1993) 

Cognitive  Task  Analysis  (CTA)  (Leplat,  1988) 

Cognitive  Task  Analysis  (CTA)  (Moray,  Sanderson  &  Vicente,  1992) 

Cognitive  Task  Analysis  (CTA)  (Neerincx  &  Griffioen,  1993) 

Cognitive  Task  Analysis  (CTA)  (Schaafstal  &  Schraagen,  1992) 

Integrated  Task  Analysis  Model  (ITAM)  (Ryder  &  Redding,  1993) 

Jackson  System  Development  (JSD)  (Jackson,  1983) 

Goals  Operators  Methods  and  Selection  Rules  (GOMS)  (Card,  Moran  & 
Newell,  1983) 

Hierarchical  Task  Analysis  (HTA)  (Annetl  &  Duncan,  1967) 

Knowledge  Analysis  of  Taste  (KAT)  (Johnson;  in  Diaper,  1989) 

Modem  Structured  Analysis  (MSA)  (Yourdon,  1989) 

Task-Action  Grammar  (TAG)  (l^yne  &  Green;  in  Diaper,  1989) 

Task  Analysis  for  Knowledge  Descriptions  (TAKD)  (Diaper,  1989) 

Task-based  Cognitive  Modelling  (Mullins  &  Treu,  1993). 

Table  II  illustrates  the  emphasis  the  techniques,  in  our  opinion,  place  upon  each 
of  the  modelling  views. 
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Table  n  Proposed  weight  (high,  mediuia,  low,  no)  of  the  analysis 
techniques  for  the  modelling  views  of  cognitive  task  analysis. 
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